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SECTION  1 


INTRODUCTION 


One  of  the  goals  in  the  development  of  Virtual  Reality  (VR)  is  to  achieve  "total 
immersion"  where  one  sees  and  interacts  with  objects  in  a  virtual  world  in  the  same  way  that 
one  sees  and  interacts  with  the  objects  of  the  real  world.  One  becomes  immersed  in  another 
world  through  the  sense  of  sight.  Thus  the  technique  of  displaying  a  virtual  environment 
determines,  in  part,  the  degree  to  which  one  can  become  transported  out  of  the  real  world  and 
into  the  virtual  world. 

The  developers  of  VR  have  utilized  the  head  mounted  display  (HMD)  as  a  means  of 
displaying  the  virtual  environment  to  the  user.  The  HMD  is  not  new  but  has  seen  widespread 
use  as  an  information  display  in  military  aircraft.  In  an  HMD,  displays  and  imaging  optics 
mounted  on  a  headset  provide  a  virtual  image  in  front  of  the  eyes.  In  the  VR  domain,  this 
design  provides  the  user  with  a  view  of  the  virtual  environment  while  blocking  out  the  user’s 
real  environment  A  tracking  device  allows  the  computer  to  present  a  viewpoint  corresponding 
to  the  user's  head  position  and  orientation,  thus  enabling  freedom  of  movement  for  the  user 
within  the  virtual  environment  These  and  other  characteristics  enhance  the  total  immersion 
capabilities  of  the  HMD. 

Several  HMDs  are  commercially  available  for  VR  applications,  including  the  Eyephone 
made  by  VPL  Research  Inc.  of  Foster  City,  CA,  the  Cyberface  made  by  LEEP  Systems  in 
Waltham,  MA,  the  Flight  Helmet  made  by  Virtual  Research  in  Sunnyvale,  CA,  and  the  Virtual 
Reality  Group  HMD,  made  in  Vienna,  VA.  Other  prototypes  have  been  developed  at  research 
centers,  such  as  University  of  North  Carolina,  the  Air  Force  Institute  of  Technology  in  Ohio, 
the  NASA/Ames  Research  Center  in  Moffett  Field,  CA,  and  the  Naval  Ocean  Systems  Center 
in  San  Diego,  CA.  Factors  that  limit  the  ability  of  these  devices  to  achieve  total  immersion 
include  the  lack  of  a  high  resolution  image,  variable  focus  optics,  and  a  wide  field  of  view 
(FOV). 
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This  report  is  an  assessment  of  the  present  state-of-the-art  in  HMD  technology  applied  to 
VR.  It  describes  ti.»  technical  capabilities  of  existing  HMD  configurations  and  evaluates  these 
characteristics  with  respect  to  immersive  VR.  We  conclude  that  major  advances  are  required  in 
HMD  technology  to  achieve  an  adequate  immersive  VR  environment  and  user  acceptance. 


1.1  SCOPE 

Closed  system  HMDs  are  evaluated  in  this  report  on  the  basis  of  quantifiable  factors 
relating  to  their  physical  performance,  such  as  FOV  and  visual  resolution.  Closed  system 
HMDs  allow  only  the  viewing  of  a  single  virtual  image,  as  opposed  to  open  system  HMDs, 
which  overlay  a  virtual  image  on  a  real  world  image.  Possible  enhancement  of  immersive  VR 
due  to  the  use  of  various  other  input/output  devices  in  conjunction  with  the  HMD,  such  as  the 
DataGlove  and  tracking  devices,  is  beyond  the  scope  of  this  paper.  Perceptual  biases  among 
individual  users  affect  the  degree  of  immersion  experienced  through  VR  [Ellis].  Evaluation  of 
these  are  also  beyond  the  scope  of  this  paper  for  obvious  reasons. 


1.2  DOCUMENT  ORGANIZATION 

The  report  is  structured  as  follows:  section  2  of  this  report  gives  an  overview  of  HMD 
design  and  delineates  the  goals  for  this  form  of  display  of  Virtual  Reality.  Section  3  gives  a 
technical  assessment  of  current  HMD  technology  mirroring  the  criteria  described  in  section  2. 
Section  4  contains  conclusions  and  recommendations  about  HMD  technology. 
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SECTION  2 


OVERVIEW  OF  HMD  DESIGN 

2.1  BASIC  STRUCTURE 

Current  VR  HMDs  consist  of  four  basic  parts:  1 )  a  display  or  image  source,  usually  a 
cathode  ray  tube  (CRT)  or  a  liquid  crystal  display  (LCD),  which  presents  an  image  to  the 
viewer.  For  binocular  viewing,  two  displays  are  required,  one  for  each  eye;  2)  an  optical 
system  that  allows  the  screen  to  be  placed  very  close  to  the  eyes  and  head  for  compactness, 
magnifies  the  image  so  it  appears  "life  size,"  and  ideally  provides  a  wide  FOV;  3)  a  head 
mount,  which  provides  a  base  for  mounting  these  components;  and.  4)  a  position  tracker, 
which  monitors  the  head  position  of  the  user.  The  computer  uses  this  information  to  generate 
an  image  corresponding  to  the  user's  head  position  and  orientation  (figure  1 ). 


2.2  VIRTUAL  REALITY  GOALS 

People  are  accustomed  to  the  way  they  already  see  the  real  world.  A  virtual  environment, 
which  parallels  the  conditions  of  sight  in  the  real  world,  will  be  both  useful  and  widely 
accepted.  Therefore,  we  establish  goals  for  the  display  quality  of  the  HMD  with  regard  to  the 
characteristics  of  normal  visual  sight.  These  are: 

1 .  Three  dimensional  (3D)/binocular  viewing 

2.  FOV 

3.  Resolution 

4.  Color 

5.  Freedom  from  distortion  and  aberrations 

6.  Accommodation  (focus)  over  a  range  of  viewing  distances 

In  addition,  the  HMD  must  be  constructed  in  such  a  manner  that  it  is  easy  and 
comfortable  to  use: 


3 


1 .  It  should  be  adjustable  for  physical  variations  between  users  -  head  size,  and 
interpupillary  distance  (IPD). 

2.  There  should  be  mLiimal  physical  restrictions  relating  to  weight  and  tethers. 

Aii  of  these  characteristics  are  both  necessary  and  desirable  for  the  HMD  to  provide  an 
effect  of  total  immersion  that  matches  the  viewer's  natural  experience.  Realizing,  however, 
that  the  reproduction  of  normal  vision  in  an  HMD  or  with  any  other  current  display  device 
might  be  beyond  the  ability  of  current  display  technology,  we  set  realistic  goals  for  the  visual 
quality  of  HMDs.  These  suggested  goals  are  a  compromise  between  those  characteristics, 
which  reasonable  advances  in  current  technology  should  be  able  to  provide  and  those  that 
humans  will  find  usable.  Table  1  compares  the  natural  human  visual  characteristics  to  the 
suggested  goals  for  HMD  visual  characteristics. 

Table  1.  Suggested  VR  HMD  Goals  versus  Human  Visual  Performance 


Binocular 

Overlap 

Total  FOV 

Resolution 

Color 

Weight 

Tethers 

Human 

Vision 

120  deg 

1  arc-min/ 
element 

greater  than 
1,000,000 

N/A 

N/A 

Realistic 

HMD 

Goals 

70  deg 

140  deg  H 
100  deg  V 

2  arc-min/ 
element 

32,768 

distinguishable 

12  ounces 

none 

2.2.1  3D/BinocuIar  Vision 

Binocular  vision  is  an  important  depth  cue.  A  slightly  different  perspective  of  an  object 
presented  to  the  right  and  left  eyes  is  fused  by  the  brain,  which  enables  us  to  perceive  an  object 
in  three  dimensions.  Depth  perception  provided  by  binocular  vision  must  be  carried  over  into 
the  virtual  environment  in  order  for  the  user  to  function  naturally  in  VR.  It  allows  the  user  to 
make  correct  judgments  about  the  relative  positions  of  objects. 
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2.2.2  Field  of  View 


The  monocular  FOV  of  the  human  eye  is  150  degrees  in  the  horizontal  and  135  degrees  in 
the  vertical  [Clapp].  The  total  horizontal  FOV  is  180  degrees  (without  head  motion).  Thus 
there  is  a  natural  binocular  overlap  of  about  120  degrees.  The  FOV  of  the  HMD  must 
^proximate  this  as  much  as  possible  for  the  use  of  the  peripheral  field  and  the  illusion  of  total 
immersion. 

2.2.3  Resolution 

The  human  eye  is  capable  of  discerning  an  element  size  corresponding  to  about  1  minute 
of  arc  at  the  very  center  of  the  person’s  gaze.  The  resolution  capability  of  the  eye  falls  off 
rapidly  with  the  angle  from  the  center  of  this  area  of  high  resolution  capability.  Assuming  that 
the  HMD  technology  allows  eye  motion  and  thus  any  point  in  a  given  FOV  could  be  the  focus 
of  a  person's  gaze,  a  50  degree  monocular  FOV  would  have  to  contain  at  least  30(X)  white  pixel 
elements  to  render  this  resolution.  Although  this  resolution  would  be  desirable,  current  display 
technology  cannot  provide  this  resolution  in  a  4-inch  diagonal  or  smaller  display,  the  size  of  the 
displays  typicady  used  in  the  HMD.  A  more  reasonable  goal  for  HMD  resolution  might  be 
2048  X  2048,  which  is  currently  achieved  with  laige  surface  CRT  technology.  On  the  other 
hand,  a  1280  x  1024  white  pixel  matiix  in  a  small  display  is  the  minimum  acceptable  resolution 
needed  to  take  advantage  of  present  commercial  workstation  image  generating  capabilities. 

2.2.4  Color 

Since  people  are  used  to  viewing  the  world  in  color,  a  full-color  display  would  add  to  the 
realism  of  the  system.  Color  can  also  be  used  for  coding  in  various  applications.  From  current 
experience,  graphics  workstations  require  a  frame  buffer  at  least  24  bits  deep  for  suitable  high 
quality  color  applications.  This  represents  over  16  million  colors.  However,  the  display  may 
not  be  ^.ble  to  reproduce  all  of  the  distinct  colors.  As  a  minimum,  15  bits  should  be  required. 
Therefore,  a  color  palette  with  32,768  colors  is  anticipated  from  the  display. 
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2.2.5  Freedom  from  Distortion  and  Aberrations 


The  environment  is  perceived  by  the  eyes  in  an  onhoscopically  correct  manner.  The 
proportions  of  viewed  objects  are  realistic  and  not  compressed,  elongated,  bent,  or  distorted  in 
some  other  way.  A  straight  line  is  a  straight  line  and  does  not  appear  to  be  anything  else.  Colors 
are  usually  continuous  over  the  object  from  which  they  are  reflected.  A  virtual  environment 
image  must  reflect  these  qualities  of  the  real  world,  free  from  geometric  distortion  and  chromatic 
aberrations,  which  would  affect  the  user’s  perception  of  an  image.  Misjudgments  that  are 
possible  in  a  distorted  environment  might  also  affect  the  user’s  performance  in  that  environment 

2.2.6  Accommodation/Convergence 

The  human  eyes  accommodate,  or  focus,  and  converge  to  the  same  point  in  space  when 
viewing  an  object  in  the  three  dimensions  of  our  natural  surroundings.  The  simultaneous 
accommodation  and  convergence  of  the  eyes  allows  one  to  comfortably  view  objects  that  range 
in  distance  from  infinity  to  the  individual's  near  point  of  accommodation,  a  distance  of  about 
0.25  meters.  Ideally,  we  would  like  to  have  the  same  viewing  range  with  an  image  provided 
by  the  HMD. 

2.2.7  Adjustable  Elements  for  User  Variation 

The  physiological  makeup  of  each  person  is  not  the  same.  Therefore,  adjustable  elements 
must  be  incorporated  in  the  design  of  the  HMD  to  account  for  differences  in  head  size,  IPD  to 
accommodate  binocular  viewing,  and  the  wearing  of  eyeglasses. 

2.2.8  Physical  Limitations 

An  ideal  HMD  for  use  in  VR  should  weigh  no  more  than  an  average  pair  of  eyeglasses 
and  be  as  easy  to  place  on  and  off.  This  is  not  within  the  realm  of  current  technology.  The 
weight  of  the  HMD  must  be  minimized  to  avoid  neck  strain,  user  fatigue,  and  gain  user 
acceptance.  An  acceptable  maximum  weight  for  the  HMD  is  about  12  ounces. 
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Free  range  of  movement  is  desired  when  working  in  the  virtual  environment  An  HMD 
should  be  constructed  without  counterweights  and  tethers  that  seriously  impede  the  user's  free 
range  of  movement 

2.3  HMD  OPTIC  DESIGN  FEATURES 

The  significant  differentiating  factor  in  HMDs  used  for  VR  is  the  type  of  optical  system 
used.  The  type  of  optics  determine  the  other  parameters  of  the  system,  such  as  FOV  and 
accommodation/convergence  range. 

2.3.1  Infinity  Optics 

The  use  of  infinity  optics  in  the  HMD  is  derived  from  the  heads  up  display  (HUD)  used 
in  military  aircraft.  In  the  HUD,  the  so  called  set  of  infinity  optics  collimate  the  rays  from  the 
image  source  to  produce  an  image  that  appears  at  infinity  (figure  2).  This  virtual  image  is  then 
overlaid  on  the  real  scene  in  front  of  the  pilot  The  advantage  of  this  approach  is  that  it 
eliminates  the  need  for  the  pilot  to  look  down  at  the  flight  instruments. 

The  application  of  infinity  optics  in  HMDs  is  the  same  as  in  the  HUD.  Rays  from  the 
display  are  collimated  to  appear  as  though  they  are  projected  from  infinity.  The  infinite  virtual 
image  allows  the  eyes  to  remain  focused  and  converged  at  infinity. 

Designs  employing  infinity  optics  in  conjunction  with  CRTs  have  been  used  in 
prototypes  built  at  Air  Force  Institute  of  Technology,  Naval  Ocean  Systems  Center  (NOSC), 
and  in  the  commercially  available  VR  Group  HMD.  The  CRTs  are  mounted  either  on  the  front 
of  the  head  directly  facing  the  eyes  as  in  the  NOSC  HMD  (figure  3),  or  on  the  side  of  the  head, 
using  relay  optics  to  project  the  image  to  the  eye,  as  in  the  VR  Group  HMD  (figure  4). 


'  VWs' 


Figure  3.  The  Naval  Ocean  Systems  Center  HMD  with  Front-Mounted  CRTs 


Figure  4.  The  VR  Group  HMD  with  Side-Mounted  CRTs.  The  Image  is  Projected  in  Front  of 

the  Eyes  by  Means  of  Relay  Optics  and  Min'ors 
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2.3.2  Magnifler  Optics 


The  other  type  of  optical  system  is  a  magnifying  system.  It  produces  a  virtual  image 
plane  somewhere  within  the  range  of  0.4  m  to  infinity,  depending  on  the  focal  lengths  of  the 
lenses  and  the  relative  placement  of  the  displays  and  lenses  (figure  5). 

Eric  Hewlett  of  LEEP  Systems  in  Waltham,  MA,  designed  a  wide  angle  set  of  optics  for 
viewing  stereoscopic  photographs  taken  with  a  stereoscopic  camera.  Essentially  a  wide  angle 
magnifier,  the  LEEP  optics  compress  most  of  the  information  into  the  central  portion  of  the 
FOV.  The  LEEP  optics  have  been  applied  to  HMDs  for  use  in  conjunction  with  LCDs.  They 
provide  a  horizontal  FOV  of  up  to  140  degrees.  Virtual  Research,  AFTT,  and  NASA  use  LEEP 
optics  in  their  HMDs.  They  are  also  used  in  LEEP  Systems'  commercial  product  called  the 
Cyberface.  Figure  6  shows  a  picture  of  Virtual  Research's  Flight  Helmet. 

VPL  has  also  developed  a  set  of  magnifying  lenses  for  their  Eyephone  HRX  after  using 
LEEP  optics  in  an  earlier  model.  They  now  use  a  set  of  Fresnel  lenses,  which  are  lighter  and 
produce  less  geometrical  distortion  than  the  LEEP  optics. 

Also  of  interest  is  Fake  Space  Lab's  Binocular  Omni-Orientation  Monitor  (BOOM).  It 
incorporates  LEEP  magnifier  optics  with  CRTs  in  a  device  that  does  not  sit  on  the  head  but  is 
mounted  to  a  moveable  arm  (figure  7). 
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Rgurc  5.  Magnifier  Optics  Produce  Virtual  Images  of  the  Display  Screens 
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SECTION  3 


TECHNICAL  ASSESSMENT 

3.1  UNDERLYING  TECHNOLOGY 

There  are  certain  factors  that  impose  limits  on  the  visual  quality  and  size  of  HMDs  that 
can  be  engineered  for  VR  use. 

3.1.1  Resolution 

Achieving  adequate  resolution  is  an  important  challenge  facing  current  HMD  designs. 

The  CRT  is  useful  because  it  currently  provides  the  highest  resolution  available,  a  1280  x  1024 
matrix  in  a  monochrome  1-inch  display.  The  physical  makeup  of  the  CRT,  however,  will  not 
allow  further  reduction  in  size  and  weight  to  make  it  applicable  for  use  in  VR  HMDs.  A 
display  slightly  less  than  1  inch  in  diameter  weighs  between  3.5  and  6  ounces  and  is  at  least  4 
inches  in  length  [Hughes].  The  LCD,  on  the  other  hand,  has  a  relatively  small  weight  (2.8  to 
6.3  ounces)  for  the  larger  3-inch  to  4-inch  screen  and  is  less  than  1  inch  thick  [SHARP 
bulletin].  The  resolution,  however,  is  no  better  than  one-third  of  the  resolution  of  the  CRT. 
Unfortunately,  there  is  not  a  large  enough  demand  for  higher  resolution  LCDs  to  warrant  their 
further  development  for  this  particular  application. 

A  possible  solution  to  the  resolution  problem  is  the  development  of  area  of  interest  (AOI) 
displays.  These  involve  the  use  of  a  high  resolution  insert  in  a  low  resolution  p)eripheral  view. 
LEEP  Systems  is  currently  developing  such  a  dual  resolution  system,  incorporating  LEEP 
Optics  with  two  different  LCD  screens  for  each  eye. 

Another  possibility  is  the  incorporation  of  Tektronix  Liquid  Crystal  Shutters.  The  system 
preserves  the  monochrome  CRT  resolution  in  a  color  image  by  using  a  field-sequential  color 
display.  The  absence  of  the  three-color  phosphor  triad  in  this  system  may  be  a  solution  to 
some  of  the  current  problems  with  resolution  under  magnified  conditions.  However,  this 
approach  would  add  more  weight  and  tethers  to  the  existing  HMD  configuration. 
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3.1.2  Accommodation/Convergence  Conflict 

Associated  with  the  binocular  viewing  of  stereoscopic  images  is  the  accommodation/ 
convergence  conflict  In  the  real  world,  the  eyes  accommodate  and  converge  to  the  same  point 
The  lack  of  conflict  in  natural  viewing  allows  us  to  view  images  clearly  over  a  large  range  of 
distances.  In  stereoscopic  viewing,  on  the  other  hand,  the  eyes  no  longer  accommodate  and 
converge  to  the  same  point  This  is  caused  by  the  fact  that  a  pair  of  two  dimensional  images  is 
perceived  as  an  image  in  three  dimensions.  The  eyes  can  comfortably  tolerate  only  a  certain 
amount  of  discrepancy  between  accommodation  and  convergence  angles  [Lipton].  This 
effectively  sets  limits  on  the  viewing  range  of  the  display  unit,  which  depends  on  the  placement 
of  the  virtual  image  plane.  The  HMD  must  be  designed  with  these  constraints  in  mind. 

The  3D  effect  is  achieved  by  presenting  a  slightly  different  view  of  an  object  to  each  eye. 
The  two  images  are  horizontally  offset  by  a  distance  that  is  referred  to  as  the  parallax  (figure  8). 
The  amount  of  parallax  determines  the  apparent  position  of  the  object  in  relation  to  the  screen. 
Negative  parallax  places  the  image  in  front  of  the  screen.  In  this  case,  the  right  and  left  images 
are  interposed.  Zero  parallax  places  the  image  on  the  plane  of  the  screen.  Positive  parallax 
places  the  images  behind  the  screen  (figure  9).  If  the  parallax  is  greater  than  the  interocular 
distance  of  the  eyes,  the  eyes  would  have  to  diverge  to  fuse  the  images.  Divergence  must  be 
avoided  because  it  is  unnatural  and  puts  strain  on  the  eyes. 

A  guideline  for  comfortable  viewing  of  a  stereoscopic  image  is  a  1.6  degrees  maximum 
change  in  convergence  angle  [Lipton].  In  figure  8,  this  change  in  convergence  angle  is  (Beta)  - 
(Alpha),  The  maximum  screen  parallax  that  gives  rise  to  the  1,6  degree  change  in  convergence 
angle  can  be  expressed  as  Pmax  =  0.03  x  Dscreen-  Pmax  is  the  maximum  screen  parallax  and 
Dscrcen  is  the  distance  between  the  eyes  and  the  viewing  plane  [Veron  et  al,  1989].  For 
viewing  objects  at  infinity,  the  viewing  plane  must  be  set  at  least  2.3  m  from  the  eyes  [Veron  et 
al,  1989].  To  view  objects  that  would  be  within  working  distance,  about  0.5  to  1  m,  the  image 
plane  would  have  to  be  at  about  0.6  m  from  the  eyes. 
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Figure  9.  Examples  of  Positive.  Zero,  and  Negative  Parallax,  and  the  Corresponding 

Positions  of  the  Perceived  Image 
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Physically,  it  is  impossible  to  overcome  the  accommodation/convergence  conflict  in  a 
binocular  optical  system  where  the  virtual  image  plane  is  at  a  fixed  distance  from  the  user.  It 
remains  a  possibility  to  develop  a  bimodal  system  where  the  image  plane  can  be  adjusted 
between  one  of  two  choices,  thus  allowing  accommodation  over  the  whole  range  of  distances 
[Veron,  personal  communication,  1992].  The  closer  image  plane  would  allow  the  user  to  work 
with  objects  up  close  in  three  dimensions.  Binocular  vision  would  provide  the  strongest  depth 
cue  in  this  mode.  A  more  distant  image  plane  would  allow  visualization  of  distant  objects.  If 
the  distance  to  the  objects  in  this  plane  is  sufficiently  large,  a  single  image  can  be  presented  to 
both  eyes,  since  the  binocular  depth  cues  are  not  effective  at  such  distances.  This  would 
increase  graphics  rendering  speeds  because  only  one  image  instead  of  two  images  has  to  be 
calculated.  Depth  perception  in  this  case  is  reliant  on  other  cues,  such  as  interposition  and 
relative  size. 

Only  through  a  variable  virtual  image  plane  system  will  the  conflict  of  accommodation 
and  convergence  be  resolved  so  that  viewing  can  take  place  over  a  normal  range  of  distances. 

A  method  of  making  the  transition  from  one  viewing  plane  to  the  other  would  have  to  be 
worked  out, 

3.1.3  Misaccommodation 

Some  problems  of  misaccommodation  and  spatial  disorientation  with  military  HUDs  and 
HMDs  have  been  documented  [Bibennan,  Hale,  Roscoej.  It  has  been  found  that  the  eye  does 
not  have  a  resting  point  of  accommodation  at  infinity,  but  instead  at  a  distance  of  about  1  m 
from  the  eyes,  varying  among  individuals.  This  is  believed  to  have  caused  problems  with 
orientation  and  spatial  judgments  in  aircraft.  The  effects  of  misaccommodation  in  relation  to 
closed  system  HMDs  like  those  in  VR  have  not  been  studied  in  detail. 

If  there  is  a  misaccommodation/spatial  misjudgmeni  problem,  it  must  be  determined 
whether  the  source  of  the  problem  is  due  to  the  physiological  makeup  of  the  user's  eyes,  or 
whether  it  is  a  problem  inherent  to  the  use  of  an  artificially  created  3D  image.  In  either  case,  it 
also  must  be  determined  whether  the  HMD  system  design  can  correct  these  problems, 
especially  if  the  problem  is  user  specific. 
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This  leads  to  another  problem:  can  VR  images  be  created  to  match  the  spatial 
relationships  of  the  real  world?  In  an  open  system,  the  overlay  of  the  virtual  image  on  the  real 
world  absolutely  requires  that  the  perceived  spatial  relationships  be  matched.  In  a  closed 
HMD,  it  would  be  desirable  to  m"*ch  the  spatial  relationships  of  the  virtual  world  to  those  of 
the  real  world,  since  the  user  is  already  used  to  his  perception  of  the  surroundings.  In  a 
training  simulation,  correct  spatial  simulation  may  be  required  if  what  is  learned  is  to  be  carried 
into  the  real  world  environment  However,  it  is  possible  that  spatial  relationships  could  be 
relearned  in  the  virtual  environment  The  problems  and  solutions  related  to  misaccommodation 
and  spatial  misjudgment  require  further  investigation  [Hale]. 

3.1.4  HMD  Physical  Configuration 

The  components  of  the  HMD-the  display  screens  and  the  optics--have  intrinsic  size  and 
weight  The  incorporation  of  these  components  into  an  HMD  limits  the  extent  to  which  the  size 
and  weight  can  be  reduced.  Thus  the  reduction  to  an  acceptable  level  of  size  and  weight  will 
have  to  be  achieved  through  reduction  in  size  and  weight  of  the  components. 


3.2  PRODUCT  ASSESSMENT 

The  characteristics  of  existing  HMDs,  which  are  discussed  below,  are  summarized  in 
table  2  for  comparison. 

3.2.1  3D/Binocular  Vision 

Most  of  the  HMDs  intended  for  VR  use  incorporate  binocular  vision  for  depth 
perception.  However,  binocular  convergence  is  vinually  the  same  from  20  feet  on  out  to  2000 
feet  [Merritt].  Therefore,  when  the  images  to  be  viewed  are  beyond  20  feet,  binocular  vision 
can  be  replaced  with  vision  of  a  single  image,  where  the  same  image  is  presented  to  each  eye. 


22 


Table  2,  Comparison  of  Specification  of  Existing  HMDs 


2 


The  image  presented  should  also  have  appropriate  depth  cues  of  perspective,  size,  and 
interposition.  The  benerits  of  providing  stereoscopic  vision  beyond  20  feet  could  be  minimal 
[Veron,  personal  communication,  1992]. 

At  close  range,  binocular  vision  is  essential  for  good  3D  perception.  The  region  of 
binocular  overlap,  where  the  two  monocular  fields  of  view  overlap,  determines  the  percentage 
of  the  total  FOV,  which  can  be  viewed  stereoscopically.  This  consideration  is  important  in  the 
design  of  the  optical  configuration  of  the  HMD  and  the  design  of  graphics  presented  to  each 
eye.  There  are  three  options  for  the  alignment  of  the  optical  axes  of  the  HMD  that  affect  the 
binocular  overlap:  parallel,  divergent,  and  convergent  (figure  10).  Infinity  optics  use  parallel 
optical  axes.  This  minimizes  the  FOV,  but  the  overlap  is  very  large,  allowing  a  large  field  for 
stereoscopic  perception.  The  VR  Group  HMD  with  its  parallel  axes  claims  to  have  a  variable 
amount  of  overlap  with  settings  at  35, 67.5,  and  100  percent.  Diverging  axes  extend  the 
binocular  field,  but  decrease  the  binocular  overlap.  The  LEEP  Systems’  Cyberface  has 
diverging  optical  axes  of  25  degrees.  The  result  is  that  the  binocular  overlap  area  is  much 
smaller. 


3.2.2  Field  of  View 

Infinity  optics  have  achieved  a  maximum  monocular  FOV  of  50  degrees  diagonal  in  the 
VR  Group  HMD.  This  results  in  a  maximum  total  FOV  of  about  82.5  degrees  for  a  35  percent 
overlap  setting.  For  VR  use,  the  FOV  of  the  infinity  optics  is  further  limited  by  two  main 
factors:  size  of  the  display  and  small  exit  pupil. 

Because  of  its  size,  usually  on  the  order  of  0.5  to  1  inch  diagonal,  the  CRT  display  is  not 
conducive  to  a  wide  FOV.  A  larger  CRT,  which  could  provide  a  significantly  larger  FOV,  is 
not  feasible  because  of  the  weight  added  to  the  HMD.  The  size  of  the  exit  pupil  of  the  optics, 
about  12  mm,  combined  with  an  eye  relief  of  about  30  mm  needed  to  accommodate  eyeglasses, 
further  decreases  the  FOV.  The  effect  caused  by  the  exit  pupil  is  the  same  as  that  achieved 
when  peering  through  a  knothole.  When  right  up  next  to  it,  the  FOV  is  large,  but  looking 


Figure  10.  Binocular  Overlap  of  Divergent,  Parallel,  and  CcHivergent  Optical  Axes 


through  it  from  a  distance  of  a  foot,  the  FOV  decreases  dramatically  (figure  1 1).  Because  of 
the  relatively  small  FOV,  infmity  optics  produce  a  tunnel  vision  effect.  This  is  felt  to  be 
undesirable  for  most  VR  applications. 

Magnifier  optics  can  be  constructed  to  easily  produce  a  wide  FOV,  as  in  the  case  of  LEEP 
optics.  The  FOV  is  enhanced  by  the  69  mm  diameter  of  the  optics.  The  LEEP  Systems 
Cyberface  uses  a  4-inch  LCD  to  make  full  use  of  the  visual  field  of  the  optics.  The  Cyberface 
obtains  a  140  degree  total  field  in  the  horizontal  direction.  Other  applications  of  the  LEEP 
optics  use  a  3-inch  LCD.  This  does  not  fill  out  the  potential  FOV  of  the  optics  [LEEP 
Systems,  Standard  Product  Guide].  Setting  the  optical  axes  parallel  to  each  other  reduces  the 
FOV  but  increases  the  binocular  overlap.  The  Virtual  Research  Flight  Helmet  with  its  parallel 
axes  achieves  a  total  FOV  of  100  degrees,  which  is  about  40  degrees  less  than  the  Cyberface  2. 

LEEP  optics  are  used  with  CRT  displays  in  Fake  Space  Labs'  BOOM  counter-balanced 
arm  device.  This  allows  the  wide  FOV  to  be  used  in  conjunction  with  the  high  resolution 
image.  The  use  of  LEEP  wide  angle  optics  with  CRT  displays  in  an  actual  HMD  is 
contraindicated  by  the  fact  that  a  small  1-inch  display  would  not  fill  up  the  visual  field  of  the 
optics. 

An  incorrect  FOV  can  also  be  presented  to  the  user  if  the  eyes  are  not  in  the  expected 
position  with  respect  to  the  optics  which  corresponds  to  the  calculated  point  from  which  the 
scene  should  be  viewed.  Because  the  eyes  will  not  always  be  in  this  expected  position,  the 
sight  lines  to  the  image  will  not  be  correct,  thus  the  scene  will  not  be  perceived  correctly 
[Robinett]. 

3.2.3  Resolution 

Current  CRTs  used  in  infinity  optics  provide  1280  x  1024  addressable  elements  in  a  1- 
inch  diagonal  monochrome  display.  The  CRT  display  used  in  the  VR  Group  HMD  yields  an 
addressable  resolution  of  2.3  arc-minutes  horizontally  by  2.9  arc-minutes  vertically  per  pixel. 
The  pixel  size  is  18  microns,  which  yields  an  apparent  size  of  only  0.09  mm  per  pixel  under 
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Figure  1 1.  Effects  of  Small  or  Large  1 
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the  5x  magnification  of  the  optics.  The  figures  for  resolution  given  in  this  section  are 
calculated  based  on  the  method  given  by  Dunn-Roberts  in  the  appendix  of  his  report.  A 
reproduction  of  this  appendix  is  given  in  appendix  A. 

A  LCD  display,  as  used  in  LEEP  Systems  Cyberface  2,  is  a  4-inch  diagonal  full-color 
LCD  that  renders  a  white  pixel  matrix  of  only  319  x  117.  The  white  pixel  size  is  0.26  mm  x 
0.53  mm.  The  resolution  given  by  LEEP  Systems  at  the  center  of  the  screen  in  radians  was 
obtained  by  dividing  the  white  pixel  size  in  mm  by  the  axial  focal  length  of  the  optics  (given  as 
42  mm  for  the  LEEP  lenses).  The  pixel  pitch,  given  as  pixel  size  in  mm  divided  by  the  focal 
length,  converted  to  arc  minutes  yields  a  resolution  of  21  arc  minutes  horizontally  by  44  arc 
minutes  vertically  per  white  pixel.  These  are  exceptionally  large.  The  pixels  at  the  edges  of  the 
screen  are  even  larger  due  to  the  nature  of  the  distortion  of  the  optics. 

The  VPL  Eyephone  HRX  boasts  an  individual  color  element  manix  of  720  x  480,  or 
approximately  416  x  277  white  pixel  elements.  In  a  calculation  by  Dunn-Roberts,  this  yields  a 
resolution  of  about  12  arc  minutes  horizontally  by  16  arc  minutes  vertically  per  white  pixel 
element  (see  appendix  A).  The  resolution  of  LCDs,  when  compared  to  the  high  resolution 
capability  of  the  eye,  represents  the  greatest  problem  with  LCD-based  HMDs.  With  a  CRT, 
the  resolution  is  enhanced  by  a  factor  of  at  least  five. 

The  magnification  provided  by  two  separate  systems  illustrate  some  of  the  problems 
involved  with  the  resolution  of  the  displays.  The  Cyberface  magnification  power  is  6.5x.  This 
yields  a  virtual  pixel  size  of  1.7  mm  x  3.5  mm  at  the  center  of  the  screen.  Toward  the  edges  of 
the  FOV,  the  pixels  are  much  larger  because  of  the  nonuniform  compression  of  the  LEEP 
optics.  With  3-inch  LCDs,  as  in  the  case  of  the  Virtual  Research  Right  Helmet,  the 
magnification  is  about  9.6x,  yielding  magnified  virtual  pixel  sizes  of  2.4  mm  x  5  mm.  The 
individual  color  pixels  are  discernible.  The  image  becomes  a  collection  of  colored  blocks  rather 
than  a  smooth,  continuous  picture.  The  images  overall  are  unacceptable  due  to  the  low 
resolution  of  the  LCDs  and  the  small  FOV  of  the  CRT  systems. 
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Another  problem  with  the  LEEP  Systems'  Cyberface  and  other  LCD  HMDs  is  that  the 
image  is  defocused  as  a  means  of  decreasing  the  granularity  (low  pass  spatial  frequency 
filtering).  The  problem  is  that  the  eyes  are  no  longer  able  to  accommodate,  thus  there  is  no 
longer  accurate  depth  perception.  Furthermore,  it  is  impossible  to  distinguish  detail  in  an 
unfocused  image. 

3.2.4  Color 

The  LCDs  currently  being  used  are  capable  of  marginally  generating  adequate  color 
combinations  for  VR  use.  Problems  arise  because  the  RGB  color  elements,  grouped  to  form  a 
white  pixel,  become  visible  under  magnification.  The  pixels  are  no  longer  fused  to  create  the 
impression  of  different  colors. 

The  same  problem  can  occur  when  a  color  CRT  image  is  magnified.  The  use  of  a 
monochrome  display  eliminates  this  problem,  but  a  monochromatic  image  is  not  desirable  for 
immersive  VR  displays. 

3.2.5  Distortion  and  Aberrations 

Wide  angle  magnifier  optics  introduce  geometric  distortion  and  chromatic  aberrations  to 
the  image  when  used  with  both  CRTs  and  LCDs.  The  LEEP  optics  induce  severe  pincushion 
distortion  (figure  12).  This  can  be  corrected  for  in  the  generated  graphics,  but  is  costly  in 
terms  of  the  required  processing.  If  the  distortion  is  ignored,  the  user  may  be  able  to  adapt  to 
it,  but  the  correct  visual  spatial  relationships  may  no  longer  be  preserved,  thus  inhibiting 
performance  in  the  virtual  environment. 

3.2.6  Accommodation/Convergence  Conflict 

The  infinity  optics  produce  an  apparent  screen  distance  of  about  7  m.  Using  the 
geometry  of  Southard's  stereoscopic  vision  system  [Veron,  et  al],  a  minimum  image 
convergence  distance  of  about  1.8  m  is  calculated.  This  is  too  great  a  distance  for  a  person  to 
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12.  LEEP  Pincushion  Distortion  of  a  Rectangular  Grid 


physically  interact  with  one's  hand  in  real  space.  The  infinity  optics  are  useful  in  cases  where 
it  is  only  necessary  to  view  distant  objects.  This  defeats  the  interactive  application  of  VR. 
unless  it  is  done  under  conditions  of  different  spatial  relationships. 

The  magnifier  optics  create  variable  image  plane  distances  according  to  the  focal  length  of 
the  optics  and  the  relative  placement  of  the  optics  and  the  screen.  The  VPL  Eyephone  has  a 
calculated  virtual  image  plane  distance  at  about  0.4  m  with  a  distance  of  16.4  mm  between  the 
optics  and  the  screens  [Robinett].  This  places  the  convergence  limits  between  0.34  m  to 
0.48  m,  a  range  which  is  good  for  close  up  work  but  not  much  else.  Virtual  Research  gives 
similar  figures  for  the  distance  to  the  image  plane  in  their  Flight  Helmet.  A  graph  (figure  13) 
shows  possible  virtual  image  plane  distances  that  can  be  achieved  with  a  LEEP  optics  system 
with  a  focal  length  of  20.7  mm.  Similar  graphs  can  be  constructed  for  lenses  of  different  focal 
lengths. 

In  a  conversation  with  Eric  Hewlett  of  LEEP  Systems,  we  obtained  some  optical 
specifications  for  their  Cyberface.  They  said  it  has  an  interaxial  optical  spacing  of  64  mm  and 
an  infinity  point  conjugate  spacing,  or  parallax  for  images  that  appear  to  be  at  infinity,  of 
62  mm.  He  also  said  the  focal  length  of  the  LEEP  optics  is  42  mm.  The  virtual  image  plane  is 
calculated  to  be  at  about  2.1  m  from  the  eyes.  The  minimum  convergence  distance  is  1.1  m, 
and  the  maximum  is  about  25  m,  effectively  infinity.  1.1  m  is  still  too  great  a  distance  to  work 
with  objects  in  the  near  field.  In  both  the  Eyephone  and  the  Cyberface,  the  comfortable  range 
of  viewing  distances  is  not  sufficiently  large  for  the  anticipated  manually  interactive  application 
in  a  VR  scene. 

Both  LEEP  Systems  and  Virtual  Research  indicated  in  private  communication  that  they 
did  not  pay  attention  to  the  conflict  in  accommodation  and  convergence  when  designing  their 
systems.  The  failure  to  do  so  inherently  restricts  the  ability  to  present  spatially  correct  images 
of  the  virtual  world  with  these  HMDs.  Mark  Bolas  of  Fake  Space  Labs  said  that  they  were  still 
experimenting  with  where  to  put  the  image  plane  in  their  B(X)M  device.  He  said  that  in 
prototypes  where  the  image  plane  distance  was  user  adjustable,  the  user  was  unable  to  place  it 
correctly.  Because  of  the  accommodation/convergence  conflict,  the  particular  application  of 
VR  will  define  an  appropriate  distance  to  the  virtual  image  plane. 
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Hgure  13.  Distance  Between  Optics  and  Screen  (Dob)  versus  Distance  to  Virtual  Image  Planes 

(Zv)  for  LEEP  Focal  Distance  of  20.7  mm 
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3.2.7  Adjustable  Elements  for  User  Variations 


InterpuDillarv  Distance  flPD) 

The  IPD  is  a  user  specific  characteristic,  which  should  be  taken  into  account  in  HMD 
design.  As  figure  14  illustrates,  the  IPD  inherently  affects  the  perception  of  depth.  Each 
person  perceives  distance  in  relation  to  his  or  her  own  IPD  and  the  subsequent  angles  for 
convergence  of  the  eyes.  Merely  adjusting  the  IPD  and  not  changing  the  parallax  for  each  user 
will  cause  inaccuracies  in  the  depth  perception.  The  IPD  should,  therefore,  be  introduced  in 
the  graphics  rendering  model  as  a  user  specific  constant  [Robinett]. 

Infinity  optics  require  an  IPD  adjustment  since  the  small  size  of  the  exit  pupil  does  not 
allow  much  translational  movement  of  the  eyes  with  respect  to  the  optical  axes  of  lenses.  The 
VR  Group  HMD  does  have  an  adjustment,  but  it  is  somewhat  difficult  to  use  and  get  the 
correct  binocular  overlap  [Veron,  memo,  1992].  In  general,  .naccuracies  in  depth  perception 
are  possible  if  only  the  IPD  is  changed  and  not  the  parallax  generated  by  the  graphics 
workstation. 

The  LEEP  optics  of  the  Cyberface  do  not  have  an  adjustable  IPD,  since  the  exit  pupils  are 
large  enough  to  account  for  variations  among  users.  This  does  not  necessarily  guarantee 
accurate  depth  perception,  however.  The  Cyberface,  by  placing  the  infinity  point  conjugate 
spacing  (parallax)  of  the  images  at  slightly  less  than  the  interaxial  spacing,  ensures  that  ihe  eyes 
will  not  have  to  diverge  to  fuse  images.  Again,  neither  the  infinity  optics  nor  the  magnifier 
optics  accounts  for  the  IPD  variation  correctly. 

Head  Size 

The  different  head  sizes  are  easily  accommodated  by  means  of  an  adjustable  strap.  The 
way  the  helmet  sits  on  the  head,  and  thus  the  orientation  of  the  tracker  is  a  potential  problem 
because  it  determines  what  image  is  displayed.  Calibration  techniques  and  procedures  ^re 
required  to  remedy  this  situation. 
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Figure  14.  Eyes  with  Different  IPDs  Perceive  the  Same  Object  Distance  with  Different 

Convergence  Angles 
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3.2.8  Physical  Limitations 


Weight 

The  weights  of  the  infinity  optics  HMDs  are  usually  greater  than  the  weights  of  the 
magnifying  HMDs.  A  CRT  slightly  smaller  than  1  inch  weighs  between  3.5  and  6  ounces 
[Hughes].  Larger  CRTs  to  increase  the  FOV  are  not  a  viable  option  because  of  the  additional 
weight  A  CRT  with  a  screen  size  of  2.6  inches  diameter  weighs  at  least  1 .5  pounds  and  is 
over  6  inches  in  length  [Hughes].  The  weight  of  infinity  optics  HMDs  ranges  from  as  low  as  3 
pounds  in  the  VR  Group  HMD  to  as  high  as  6  pounds  in  other  prototypes.  The  bulk  of  the 
CRTs  contributes  greatly  to  the  moment  of  inertia  and  thus  to  user  fatigue. 

The  LCD  provides  an  obvious  advantage  in  terms  of  weight,  ranging  from  80  g  (2.8 
ounces)  for  a  3-inch  LCD  to  180  g  (6.3  ounces)  for  a  4-inch  LCD  [SHARP  product  bulletin]. 
The  weight  of  an  LCD-based  HMD  ranges  from  about  2.5  pounds  to  3.7  pounds. 

Mounting  locations  of  the  displays  and  optics  is  also  a  concern.  Current  CRT  systems 
mount  the  CRTs  on  the  side  of  the  head,  using  relay  optics  to  project  the  image  to  the  eye.  The 
optics  add  weight  to  the  system.  If  the  CRTs  were  mounted  in  front  of  the  eyes,  there  would 
be  too  much  weight  on  the  face  as  well  as  an  increase  in  moment  of  inertia  of  the  system.  A 
counterbalance  placed  at  the  back  of  the  head  further  increases  the  moment  of  inertia.  LEEP 
Systems  use  a  front  mount  for  the  optics  and  LCDs.  The  weight  is  redistributed  to  the  top  of 
the  head  from  the  face  by  means  of  a  counterweight.  Experience  has  shown  that  this  does  not 
do  enough  to  relieve  the  weight  of  the  system. 

The  weight  of  current  systems  is  still  too  great  to  permit  comfortable  use  over  an 
extended  period  of  time.  One  viable  alternative  is  mounting  the  display  unit  on  a  mechanical 
arm  as  in  the  BOOM.  This  allows  the  weight  to  be  supported  by  the  mechanical  arm  rather 
than  the  user's  head. 
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Freedom  of  Movement 


The  HMDs  can  operate  in  6  degrees  of  freedom  because  of  the  various  tracking  systems. 
The  range  of  the  tracker  is  currently  limited  to  a  circle  of  5-foot  radius  from  the  source.  The 
tethers  needed  to  drive  the  electronics  and  the  tracker  not  only  contribute  largely  to  the  weight 
bom  by  the  user  but  also  severely  curtail  the  freedom  of  movement  of  the  user. 

Counterweights  added  to  balance  the  HMD  on  the  head  conuibute  to  the  inertia  of  the 
unit.  This  decreases  the  ease  with  which  the  user  can  move  around  with  the  HMD  and  limits 
the  positions  that  can  be  comfortably  maintained. 
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SECTION  4 


CONCLUSIONS  AND  RECOMMENDATIONS 


While  HMDs  are  potentially  the  most  effective  means  of  providing  a  state  of  total 
immersion  in  VR,  the  effectiveness  of  the  HMD  is  Umited  by  the  current  state  of  display 
technology  and  the  laws  of  physics.  We  find  that  HMDs  fall  short  in  the  following  areas: 

1 .  ability  to  simultaneously  achieve  an  image  with  a  wide  FOV,  high  resolution,  and 
color; 

2.  achievement  of  an  acceptable  weight; 

3.  elimination  of  tethers; 

4.  range  of  visual  accommodation;  and 

5.  perceptual  differences  due  to  individual  physiological  characteristics  (e.g.,  IPD). 

A  breakthrough  in  HMD  technology  is  needed  to  make  it  a  viable  means  for  providing 
immersive  VR.  The  present  deficiencies  in  optical  performance  will  not  induce  the  user  to 
sustain  the  inconveniences  of  the  weight  and  tethers  of  either  optical  system.  Further 
development  of  existing  technology  (e.g.,  increasing  the  resolution  of  miniature  LCDs)  may 
provide  no  more  than  incremental  increases  in  the  quality  of  the  displays.  Moreover,  the  small 
market  demand  for  such  technology  is  unlikely  to  warrant  its  development. 
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New  ways  of  displaying  VR  in  an  HMD  need  to  be  pursued.  Possibilities  include  the 
direct  projection  of  an  image  onto  the  retina  with  a  laser.  It  should  be  determined  whether 
holographic  optical  elements,  which  have  a  greater  degree  of  refractive  ability,  can  provide 
significant  advantages  over  existing  HMD  optical  configurations.  Projecting  a  real-time 
holographic  image  from  an  HMD  by  utilizing  real-time  holographic  video  techniques  developed 
by  MIT  [Benton]  should  also  be  considered  for  immersive  VR.  The  future  of  immersive  VR 
depends  on  developing  a  method  of  displaying  the  images  that  can  achieve  the  goals  set  forth  in 
this  document. 
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APPENDIX  A» 


COMPUTATIONS  CONCERNING  EYEPHONE  RESOLUTION 


The  low  resolution  Eyephone  has  442  x  238  primary  color  pixels  per  eye.  The  total 
number  of  primary  color  pixels  is  105,196.  This  means  35,065  triads,  in  an  array  that  1  am 
assuming  is  256  x  137.  The  FOV  for  each  eye  is  86  degrees  (horizontal)  by  76  degrees 
(vertical).  The  total  horizontal  FOV  is  108  degrees.  The  binocular  overlap  is  64  degrees. 

Because  of  the  assumptions  made  above,  the  following  calculations  are  approximate. 

The  total  number  of  pixels  (three  color)  horizontally  is  256  x  108/86  (pixels  per  eye  x  total 
FOV/FOV  per  eye).  This  equals  321  pixels.  The  number  of  arc  minutes  per  pixel  horizontally 
is  108  X  60/321  or  approximately  20  arc  minutes  per  pixel. 

The  number  of  vertical  three-color  pixels  is  approximately  137.  The  number  of  vertical 
degrees  is  76.  This  gives  approximately  33  arc  minutes  per  pixel  vertically. 

The  same  calculations,  with  the  same  assumptions,  applied  to  the  high  resolution 
Eyephones  gives  the  following  results: 

Total  number  of  primary  color  pixels  per  eye  is  345,6(X)  in  a  720  x  480  matrix.  This 
gives  about  416  horizontal  by  277  vertical  three-color  pixels  per  eye.  The  horizontal  FOV  per 
eye  is  86  degrees,  and  for  both  eyes  is  106  degrees.  The  vertical  FOV  is  75  degrees.  The  total 
number  of  horizontal  pixels  is  approximately  513  pixels.  This  works  out  to  about  12  arc 
minutes  per  pixel  horizontally.  The  number  of  three  color  pixels  vertically  is  about  277.  This 
works  out  to  about  16  arc  minutes  per  pixel  vertically. 


*  Dunn-Roberts,  R.,  K.  Uliano,  P.  Moskal,  and  M.  Moshell,  "Head  Tracking  and  Head 
Mounted  Displays  for  Training  Simulation,"  Final  Report,  Visual  Systems  Laboratory 
Document  VSLM92,4,  Institute  for  Simulation  and  Training,  University  of  Central  Rorida, 
IST-TR-92-12,  5  February  1992. 
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